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sample;  the  results  were  statistically  significant.   Four 
of  the  prediction  equations  derived  were  selected  to  be 
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I.   INTRODUCTION 

A.  PURPOSE  OF  STUDY 

The  purpose  of  this  study  was  to  assess  the  character- 
istics of  the  U.S.  Navy  officer  students  registered  in  the 
Operations  Research/System  Analysis  curriculum  at  the  U.S. 
Naval  Postgraduate  School  by  means  of  a  Biographical  Ques- 
tionnaire, the  Graduate  Record  Examination  and  the  Strong 
Vocational  Interest  Blank  and  determine  if  these  measures 
offered  a  means  of  improving  the  selection  of  U.S.  Naval 
officers  for  enrollment  in  the  Operations  Research/Systems 
Analysis  curriculum. 

This  study  was  a  portion  of  the  NPS  Student  Selection 
Project  sponsored  by" the  Navy  Personnel  Research  and  Develop- 
ment Center,  concerning  the  prediction  of  officer/student 
academic  performance  and  satisfaction  in  graduate  curricula 
at  the  U.S.  Naval  Postgraduate  School. 

Only  academic  performance  was  investigated  in  the  study 
since  two  previous  studies,  conducted  by  Cook  [1]  and  Sofge 
[2],  found  that  none  of  the  predictors  of  satisfaction 
cross-validated  at  a  statistically  significant  level. 

B.  U.S.  POSTGRADUATE  SELECTION  PROCESS 

The  Postgraduate  Education  Selection  Board  in  the  Navy's 
Bureau  of  Personnel  (BuPers)  is  in  charge  of  selecting  Navy 
Officers  best  suited  for  courses  in  the  postgraduate  program 


based  on  the  individual's  preference,  for  postgraduate  curri- 
cula, professional  performance,  prior  academic  performance, 
promotion  potential  and  the  needs  of  the  service  [3] . 

The  needs  of  the  service  are  determined  by  the  Chief  of 
Naval  Personnel  and  sent  to  the  Board  annually. 

An  officer's  prior  academic  performance  is  translated 
into  an  Educational  Potential  Code  (EPC)  based  upon  an  evalua- 
tion of  previous  grades.   The  EPC  is  supposed  to  be  weighted 
40%  by  the  Board. 

An  officer's  professional  performance  is  supposed  to  be 
weighted  60%.   The  Officer's  Fitness  Reports  are  used  as 
the  primary  indicator  of  professional  performance  [3]. 

Upon  completion  of  the  selection  process,  the  Board  sends 
a  list  of  principle  and  alternate  selectees  to  the  Officer 
Detailing  Section  of  BuPers.   The  Detailers  then  decide  which 
of  these  selected  officers  are  available  for  education.   A 
more  detailed  description  of  this  selection  procedure  is 
provided  by  R.S.  Elster  [3]. 

C.   DEFINITION  OF  KEY  TERMS 

1.   The  Quality  Point  Rating  (QPR) ,  is  a  student's 
weighted  grade  score  computed  from  standards  established  by 
the  Naval  Postgraduate  School  [4]  as  follows: 


Grade  Point  Value 

A  4.0 

A-  3.7 

B+  3.3 

B  -  3.0 

B-  2.7 

C+  2.3 

C  2.0 

C-  1.7 

D+  1.3 

D  1.0 

X  0.0 


Multiplying  the  credit-hours  value  of  a  course  by  the 
point  value  received  provides  the  total  quality  points  for 
that  course.   Adding  the  quality  points  for  all  courses  and 
dividing  by  the  total  number  of  credit  hours  results  in  a 
figure  defined  as  a  Quality  Point  Rating  (QPR) .   For  example, 
if  a  student  received  four  hours  of  A,  four  hours  of  A-  and 
four  hours  of  B,  his  QPR  would  be: 


4  X  4  +  4  X  3.7  +4X3    -  c_ 
4  +  4  +  4 


2.  The  Graduate  Record  Examination  is  a  widely  accepted 
paper  and  pencil  aptitude  test  designed  to  predict  academic 
potential  at  the  graduate  level.   The  GRE  is  a  secure  test 
which  is  administered  under  controlled  conditions  and  yields 
two  scores:   verbal  ability  (GREV)  and  quantitative  ability 
(GREQ)  [5].   It  is  prepared  and  published  by  the  Education 
Testing  Service  of  Princeton,  New  Jersey. 

3.  The  Strong  Vocational  Interest  Blank  (SVIB) ,  is  a 
test  to  assess  individual  interests  as  related  to  interests 


of  incumbants  in  various  occupations  [6].   The  SVIB  contains 
399  items,  listing  school  subjects,  hobbies  and  activities 
to  which  the  individual's  response  is  either  like,  dislike, 
or  indifferent. 

4.  Biographical  Questionnaire.  This  questionnaire  was 
designed  to  obtain  historical/biographical  facts  from  officers/ 
students  as  well  as  attitudes  toward  postgraduate  education 
and  its  place  in  today's  U.S.  Navy.  The  questionnaire  con- 
tained 61  questions  which  required  "yes"  or  "no"  responses, 
and  was  developed  by  R.A.  Weitzman  with  the  assistance  of 

J.L.  Cook  [1] . 

5.  Correlation  Coefficient.   The  Simple  Correlation 
Coefficient  is  a  measure  of  linear  association  between  two 
random  variables  Y  and  X  [7] .   The  Multiple  Correlation 
Coefficient  is  a  measure  of  linear  association  between  a 
random  variable  Y  and  a  set  of  random  variables 

X  =  (X,  ,  X^  /  .  .  .  f^n'  L  7  J  . 

The  range  of  possible  values  of  the  Correlation  Coeffi- 
cient is  from  -1.00  to  +1.00.   A  correlation  of  0.00  indi- 
cates that  knowledge  of  the  value  of  X  gives  no  information 
about  Y. 

6.  The  Reliability  Coefficient,  is  a  measure  of  consis- 
tency or  stability  of  test  results.   It  is  often  obtained 

by  inter-correlating  the  test  with  the  results  if  the  test 
is  retaken  in  the  same  or  alternate  form,  or  it  is  obtained 
by  splitting  the  test  into  two  parts  and  intercorrelating  the 
two  sets  of  scores  [8]. 
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7.  Validity  is  the  effectiveness  of  the  test  in  repre- 
senting, describing  or  predicting  the  attribute  that  the 
test  purports  to  measure  [8], 

8.  Cross-Validation:   Testing  out  of  a  set  of  items  or 
system  of  test  weights  derived  from  one  sample,  to  see  to 
what  extent  the  procedure  or  equation  retains  its  validity 
with  a  new  and  independent  sample.   Cross-Validation  is 
especially  important  when  items  or  test  weights  have  been 
chosen  from  a  large  number  of  possible  alternatives,  and  when 
the  original  sample  was  small  [8]. 

D.  ASSUMPTIONS 

It  was  assumed  that  academic  performance,  represented  by 
Quality  Point  Rating  is  related  to  some  predictors,  such  as 
biographical  items,  prior  academic  performance,  SVIB  scores 
or  GRE  scores. 

E.  LIMITATION 

This  study  was  limited  to  male,  U.S.  Navy  Officer  Students 
who  were  enrolled  in  the  Operations  Research/Systems  Analysis 
curriculum  in  Spring  1974. 
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II.   DESCRIPTION  OF  INSTRUMENTS 

A.   GRADUATE  RECORD  EXAMINATION  (GRE) 

This  test  is  used  in  efforts  to  determine  student's  apti- 
tudes for  postgraduate  study.   Included  in  the  test  are  ver- 
bal reasoning  questions,  reading  comprehensive  questions  and 
various  kinds  of  mathematical  problems  involving  arithmetic 
reasoning,  algebra  and  the  interpretation  of  graphs,  diagrams 
and  descriptitve  data  [5] .   Established  reliabilities  for 
the  test  are:  [9] 

Test  Reliability  Coefficient 

Verbal  0.90 

Quantitative  0.89 

The  GRE  has  increased  in  utilization  as  a  selection  tool. 
There  were  22,000  candidates  tested  in  1958  and  the  number 
has  increased  since  then.   Over  300,000  candidates  were 
tested  in  1973  [10] . 

Previous  studies  of  the  validity  of  the  GRE  in  predicting 
academic  performance  at  the  U.S.  Naval  Postgraduate  School 
were  conducted  by  Kauder  and  Ebert  in  1963,  Dreese  and  Russel 
in  1964,  and  Cook  in  1974.   The  first  two  studies  obtained 
over  0.45  validation  correlations  while  the  later  one  obtained 
a  correlation  of  0.37  with  a  relatively  small  sample  size 
(N  =  30)  . 
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B.   STRONG  VOCATIONAL  INTEREST  BLANK  (SVIB) 

The  primary  use  of  the  Strong  Vocational  Interest  Blank 
is  for  counseling  high  school  and  college  students  about 
their  career  choices.   However  this  test  can  also  be  used 
for  personnel  selection.   When  used  by  a  professionally 
trained  person,  the  SVIB  can  provide  useful  information  for 
evaluating  an  applicant  [6] . 

The  basic  SVIB  documents  are  a  test  booklet  containing 
the  items,  an  answer  sheet  where  an  individual  records  his 
responses,  and  the  profile  forms  that  are  used  to  report 
the  results. 

The  SVIB  has  two  types  of  scales,  occupational  scales  and 
non-occupational  scales.   People  in  different  occupations 
have  different  interests.   The  SVIB  is  a  device  to  identify 
such  differences  among  those  occupations.   The  SVIB  scores 
are  used  to  counsel  individuals  regarding  occupations  in 
which  they  would  be  satisfied.   The  SVIB  accomplishes  this 
by  providing  an  index  of  the  similarity  between  a  person's 
interests  and  those  of  men  (or  women)  in  each  of  a  wide 
range  of  occupations.   Basically,  the  technique  used  in  SVIB 
is  to  present  the  individual  with  a  long  list  of  activities 
and  ask  him  to  indicate  which  he  likes  and  dislikes.   The 
answers  are  analyzed  by  comparing  his  responses  to  those  of 
men  (or  women)  already  established  in  a  wide  range  of  activi- 
ties.  If  his  choices  coincide  with  say,  engineers,  then  he 
received  a  high  score  on  the  engineers  scale.   The  mean  score 
for  a  particular  occupation  has  been  set  at  50  with  a  standard 
deviation  of  10. 
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The  Academic  Achievement  scale  (AACH)  as  one  of  the  non- 
occupational scales  is  an  attempt  to  identify  the  pattern  of 
interests  associated  with  good  scholarship.   The  scale  inclu- 
ded items  that  differentiate  between  good  and  poor  students. 
It  has  been  shown  to  be  moderately  effective  in  predicting 
grades  [6] . 

C.   BIOGRAPHICAL  QUESTIONNAIRE 

The  questionnaire  was  developed  to  obtain  the  following 
information: 

1.  Source  of  commission  and/or  prior  enlisted 
service . 

2 .  Rank . 

3.  Undergraduate  institution  attended,  degree 
received,  undergraduate  major  and  grade  point 
average . 

4.  Warfare  specialty. 

5 .  Race . 

6.  Height  and  weight. 

7.  Religion. 

8.  Marital  status  and/or  sex  dependents. 

9.  Birth-order  of  subject. 

10.  Educational  level  of  father. 

11.  Military  career  of  father. 

12.  High  school  background  information. 

13.  Personal  habits. 

14.  Participation  in  Boy  Scout  and  rank  attained. 

15.  Designation  by  Postgraduate  Selection  Board. 
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16.  Satisfaction  with  the  Naval  Postgraduate 
School  and/or  curriculum. 

17.  Possible  future  use  of  skills  obtained  at  the 
Naval  Postgraduate  School. 

18.  Mathematics  background. 

19.  Motivation  for  coming  to  the  Naval  Postgraduate 
School . 

The  questionnaire  utilized  a  booklet  with  61  questions, 

each  of  which  required  "yes"  or  "no"  responses.   The  responses 

were  recorded  on  a  separate  sheet.   The  answer  sheet  also 

contained  an  administrative  section  which  required  information 

such  as:   Social  Security  number  (for  identification) , 

current  curriculum,  number  of  quarters  completed  in  current 

curriculum,  and  QPR. 
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III.   METHODS,  TECHNIQUES  AND  PROCEDURES 

A.  GENERAL 

Seventy-two  U.S.  Navy  officer  students  in  the  Operations 
Research/Systems  Analysis  curriculum  at  NPS  who  were  at 
various  stages  in  their  curriculum  when  the  data  were  collected 
(quarter  3,  Academic  Year  73-74)  constituted  the  population 
of  this  study.   A  random  sample  of  50  students  was  randomly 
selected  from  the  population  for  the  development  of  predic- 
tors of  academic  performance.   The  remaining  sample  constituted 
the  cross-validation  sample. 

B.  LINEAR  REGRESSION  ANALYSIS 

The  purposes  of  Linear  Regression  are  to  select  the  set 
of  explanatory  variables  which  should  be  included  in  a  model, 
and  then  to  estimate  the  regression  coefficients  which 
describe  the  quantitative  weights  of  these  variables. 

The  model  to  be  used,  in  matrix  notation,  has  the  form: 

Y  =  XB  +  e 


where 


bJ 


I  e. 


Y  = 


B 


e  = 


IY  «J 

n 


B  4 
P 


n 


16 


X  = 
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X 


11 


X 


12 


X21    X22 


X  ,    X  . 
nl    n2 


X 


'IP 

2p 


np 


It  was  assumed  that 

1.  E(e)  =  0,  hence  E ( Y)  =  XB 

2.  The  components  of  e  are  identically  and  indepen- 
dently distributed,  hence  the  variance  covariance 
matrix  for  the  vector  variables  e  is 

E(e  e')  =  C2I   . 

nxn 

The  Gauss-Markov  theorem  states  that  "the  least  squares 
estimators  have  minimum  sampling  variance  in  the  class  of 
all  unbiased  estimators  which  are  linear  functions  of  Y .  " 
To  obtain  the  least  square  estimators,  one  minimizes  e'e, 
where  e  =  Y  -  XB,  with  respect  to  B.   This  yields: 

B  =  (X*X)   X'Y  , 

where  B  is  the  estimator  of  B,  provided  that  (X'X)  is 
non-singular. 

To  test  the  hypothesis  of  B  =  B*  against  the  alternative 
hypothesis  B  ^  B* ,  one  uses  Analysis  of  Variance  criteria. 
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All  Analysis  of  Variance  tests  are  based  on  what  is  known 
as  the  Likelihood  Ratio  test. 
In  the  case  of  the  model 

Y  =  XB  +  e  , 

we  need  another  assumption,  that  is  e  is  distributed  as 

2 

multivariate  normal  with  mean  0_  and  variance  C  I,  hence 

Y  ~  N(XB,  C2I) 

and  the  likelihood  function  has  the  form: 

-1 


(Y  -  XB)  '  (Y  -  XB) 


2  — 

2       1       1C 
L(Y,B,0  =    X  n/?  e 


Consider  maximizing  the  likelihood  function  with  respect 

2  ... 

to  B  and  C  under  two  situations.   The  first  case  is  when 

o 

we  allow  B  and  C   to  take  on  any  value  they  wish  in  the  range 

2 
(-°°  _<  B  <_   °°;  C   >^  0)  .   Denote  this  range  U.      This  is  the 

unrestricted  case  and  the  maximum  value  of  the  likelihood 

function  will  be  denoted  L(ft) .   The  second  case  is  when  the 

2 

restriction  on  the  parameters  B  =  B * ;  C  ^.0.   This  range  is 

denoted  by  W  and  the  maximum  value  of  the  likelihood  function 
under  the  restriction  of  the  hypothesis  is  denoted  L(W) .   We 
would  reject  the  hypothesis  B  =  B*  in  favor  of  alternative 
hypotheses  B  =  B*  if  in  fact: 


L  =  ^-<  k 
L(fi) 
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for  some  constant  k  <  1  where  k  is  chosen  such  that  the 
probability  of  rejecting  the  hypothesis  when  it  is  true  is 
equal  to  a  (usually  a  =  5  or  10%) .   That  is,  select  k  such 
that 


00 

a  =   /  g(L|Ho  is  true)  dL 

— oo 


where  g(L|Ho  true)  is  the  distribution  of  the  likelihood 

ratio  when  Ho  is  true. 

2  2 

Maximizing  L(Y,B,C  )  with  respect  to  B  and  C   under  Q 


and  W  gives  the  likelihood  ratio 


(Y  -  XB)  ■  (Y  -  XB)    n/2 

L  =  [ ] 

(Y  -  XB*) ■ (Y  -  XB*) 


-1 
with  B  =  (X'X)   X'Y. 

The  numerator  of  the  likelihood  ratio  (ignoring  the 

power  n/2)  may  be  expressed  as 


where 


(Y  -  XB*) (I  -  XS~1XI) (Y  -  XB*)  , 


S"1  =  (X'X)"1 
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The  denominator  of  the  likelihood  ratio  may  be  expressed 
as: 

(Y  -  XB*) ■ (I  -  XS~1X') (Y  -  XB*)  +  (Y  -  XB*) ' (XS_1X) (Y  -  XB*) 


Hence,  L  may  be  written  with  Z  =  (Y  -  XB*)  as 


Z'A  Z      n/2 

L  =  [ i ] 

Z'A  Z  +  Z'A2Z 


where 


A1   =  (I  -  XS   X') 


A2  =  (XS  1Xt) 


and  it  also  can  be  proven  that  A.,  and  A~  are  idempotent 

2  2 

matrices  (A   =  A, ,  A2   =  A2)  with  rank  (n  -  p)  and  p 

respectively. 

Since  Y  -  N(XB,C2I)  then  Z  ~  N (XB  -  XB*,  C2I)  and  also 
since  A,  and  A2  are  idempotent  and  A, A2  =  0  then 

Z'A^  Z'A2Z 

— 5 —        and        — 5 — 
C  C 

are  independent  Chi-Squares: 


— n —  ~  X  (n  -  p) 
C 
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and 


Z'A2Z       2 

— 5 ~  X  (p;*)  , 

where  X  is  the  non  centrality  parameter  and  its  value  is 
equal  to 


X  =  ^  (B  -  B*)  'X'XCB  -  B*)  . 


Note,  that  although  the  ratio 


Z'A^/C2 


2 1 

Z'A, Z/C   +  Z'A0Z/C 

is  not  the  ratio  of  two  independent  Chi-Squares  and  hence 
not  distributed  F,  the  statistic 

(Z'A9Z/C2)/p 
U  =  - 


(Z'A1Z/C2)/n-p 


is  the  ratio  of  two  independent  Chi-Squares  divided  by  their 
respective  degrees  of  freedom.   Moreover,  the  denominator  is 
distributed  as  central  Chi-Square  and  the  numerator  is  dis- 
tributed as  non  central  Chi-Square  and,  hence,  the  ratio  is 

distributed  as  F(p,n-p,X).   If  the  hypothesis  B  =  B*  is  true, 

1         X  'X 
the  noncentrality  parameter  X  =  -j(B   -   B*) — ~-(B  -  B*)  is  zero 

z   -       -       Qz.   - 

and  the  U  statistic  is  distributed  as  central  F(p,n-p). 
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To  show  that  the  F  test  and  the  likelihood  ratio  test 
are  equivalent,  we  note  that 


L  =  [■ 


Z'A^ 


Z'AjZ  +  Z'A2Z 


n/2 

-]     =  [- 


n/2 


1  + 


n-p 


U 


U  and  L  are  monotonically  related  in  a  manner  given  in  the 
graph  below. 


The  likelihood  ratio  test  is  said  to  reject  Ho  when  L  <  k 
where  k  is  chosen  to  give  a  significant  test  of  level  a.   But 
this  is  identical  to  rejecting  Ho  when  U  >  k'  where  k'  is 
chosen  to  give  a  significant  test  of  the  same  level  and 
since  we  know  the  distribution  of  U  when  Ho  is  true  we  can 
select  k'  from  the  appropriate  F  table  [11]. 

In  many  situations,  there  is  not  sufficient  information 
about  the  order  of  importance  of  the  independent  variables 


22 


X.  .  ..,  X   in  predicting  the  dependent  variable  Y.   Testing 

Ho:   B.  =  0  for  each  variable  X.  does  not  reveal  this  ordering 
1  1 

For  instance,  rejecting  the  Ho  that  B.  =  0  while  accepting 
the  remaining  B.  =  0  could  have  led  to  the  false  conclusion 
that  X,  was  the  only  variable  of  importance  in  predicting  Y 
[7]. 

One  solution  is  the  Stepwise  Regression  procedure  which 
selects  a  best  subset  of  predictors  according  to  the  following 
procedures:   The  first  step  selects  the  single  variable  which 
best  predicts  Y.   The  second  step  finds  the  variable  which 
best  predicts  Y,  given  the  first  variable  entered.   In  the 
steps  that  follow,  either  a  variable  is  entered  which  best 
improves  the  prediction  of  Y  given  all  the  variables  entered 
from  the  previous  step,  or  a  variable  is  removed  from  the 
set  of  predictors  if  its  predictive  ability  falls  below  a 
given  level  [7] .   The  process  is  terminated  when  no  further 
variable  improves  the  prediction  of  Y. 

C^  JACKKNIFING 

Least  square  estimations  provide  unbiased  estimators  as 
long  as  the  specified  model  is  correct.   But,  at  the  early 
stages  of  regression  the  model  is  not  complete.   Hence  the 
estimators  may  be  biased.   Generally  this  bias  diminishes 
as  the  sample  size  increases.   One  way  to  eliminate  this  is 
to  use  the  Jackknife  procedure. 

The  rationale  behind  the  Jackknife  is  as  follows  [12] : 
Let  e,  a  biased  estimate  of  a  parameter  9,  decrease  linearly 
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with  sample  size  (a  common  situation.)  .   Suppose 

A  B  A 

E(0)  =  6  +  rr—  where  Nm  =  sample  size.   Let  0   be  an  estimate 
Nm  o 

of  8  based  on  all  Nm  observations.   Let  6-,  /  0~  ,  . .  .  ,  0N  be  a 
corresponding  estimate  based  on  (N-l)m  observations,  and 


E(0.)  =  0  + 


i'    w    (N-l)m  * 


Let 


Ji(e)  =  ng  -  (N-i)ei 


Then 


E(Ji(0))  =  NE(0)  -  (N-l)E(8i) 


=  N(0+A)  „  (N-lJlei^) 


Ne  +  \  -   (N-i)  -  I 

m  m 


=  0 


N 
J(0)  =  \      E  J.  (0) 
i=l  1 


is  the  best  unbiased  estimate  of  0  with  variance  equal  to 


V(J(0))  =  i  V(J. (0)) 
N     l 


and 
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asymptotically 
J(6)  -         N(9,V(J(6))  . 


For  testing  Ho:   8  =  0  vs  Ha:   0^0,  use  the  t  statistic 


t  =      6      _       J(0) 


V(J(0>>    \T V(J.(6))/N 


i 

The  Jackknife  procedure  has  the  advantages  of  being  usable 
on  a  relatively  small  data  base  and  of  theoretically  adjusting 
for  bias. 

In  applying  this  method  to  multiple  regression,  divide 
the  sample  data  into  N  subsamples  of  equal  size  m.   Based 
on  the  Stepwise  Regression,  investigate  the  first  variable 
to  enter  into  the  equation  and  obtain  N  estimates  of  the 
parameters  and  the  corresponding  pseudo-value  J. (6)  and  J (6) 
and  use  the  t  statistic  to  test  the  hypothesis  regression 
coefficient  is  equal  to  zero  against  the  alternative  hypothe- 
sis that  the  regression  coefficient  is  not  equal  to  zero. 
If  we  reject  the  Ho,  the  next  step  would  be  to  investigate 
the  second  variable.   Repeat  the  same  procedure  until  we 
reject  the  Ho  of  including  a  variable  into  the  equation  or 
alternately  all  variables  to  be  considered  are  accepted  at 
a  statistically  significant  level  to  be  included  into  the 
equation. 

Final  estimates  of  the  parameters  are  obtained  from 
the  next  to  final  step  or  final  step  if  all  variables 
considered  are  included  in  the  equation. 
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D.   PATTERN  ANALYSIS 

Identifying  responses  to  items  as  correct  or  incorrect 
makes  it  possible  to  determine  test  scores  by  counting  the 
number  of  items  answered  correctly.   However,  the  total  num- 
ber of  items  answered  correctly  does  not  exhaust  all  the 
information  contained  in  the  responses.   Different  indivi- 
duals may  answer  different  combinations  of  items  correctly 
although  the  total  number  of  items  answered  correctly  is  the 
same.   To  each  of  these  patterns,  there  corresponds  a  differ- 
ent possible  score,  called  a  pattern  score.   On  a  three-item 
test,  for  example,  there  are  8  different  possible  pattern 
scores.   In  contrast,  there  are  only  4  different  scores 
determinable  by  counting  the  number  of  items  answered  correct- 
ly.  By  creating  a  more  refined  and  extensive  measurement 
scale,  pattern  scores  may  produce  higher  validities  than 
traditional  total-correct  scores  [13]  . 

Pattern  scores  are  also  possible  for  different  patterns 
of  any  binary  responses,  including  yes  or  no  responses  to 
items  on  a  biographical  inventory. 

A  computer  program  was  developed  by  B.F.  Folce  Jr.  [14] 
for  his  Master's  thesis  under  the  guidance  of  R.A.  Weitzman. 
It  is  called  Pattern  Analysis  Item  Nominator  (PAIN) .   PAIN 
operated  by  computing  a  mean  criteria  score  for  each  pattern 
of  responses  in  a  given  subset,  assigning  these  scores  to 
subjects  having  that  pattern  of  responses,  and  correlating 
the  assigned  scores  with  the  subject's  actual  criterion  score, 
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IV.       DATA    PRESENTATION 

A.  GENERAL 

The  primary  analysis  of  the  data  was  conducted  using 
three  packaged  computer  programs  at  the  W.R.  Church  Computer 
Center  at  the  Naval  Postgraduate  School.   These  programs  were 
the  Statistical  Package  for  the  Social  Sciences  (SPSS)  to 
provide  Stepwise  Multiple  Regression  as  well  as  Pearson 
Correlations,  BIMED03R  to  provide  Multiple  Regressions  for 
the  data  within  each  selection  of  subsamples  from  the  same 
population,  and  SNAP/IEDA  to  obtain  scatter-plots  of  the  data 

Two  programs  written  by  B.F.  Folce  were  used  in  Pattern 
Analysis,   The  first  program  called  Pattern  Analysis  Item 
Nominator  (PAIN)  provides  the  following  information: 

1.  Validities  of  all  item  subsets  examined. 

2.  A  list  of  the  items  that  form  the  most  valid 
subset  of  a  given  size. 

3.  The  validities  of  the  most  valid  subset  of  each 
size. 

The  second  program  of  Folce' s  was  used  to  cross-validate 

the  items  selected  by  PAIN. 

B.  CRITERIA  AND  PREDICTORS 
1.   Criteria 

The  performance  criterion  used  in  this  study  was 
standardized  QPR,  denoted  by  ZQPR.   This  standardization  was 
made  to  ensure  that  the  differences  among  students  would  be 
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due  to  differing  individual  performances,  eliminating  possible 
effects  related  to  quarters  completed  in  the  curriculum.   It 
was  derived  by  computing  the  mean  QPR  and  its  standard  devia- 
tion for  each  input  group.   Individual  ZQPR,  then,  was  com- 
puted using  its  mean  and  standard  deviation  according  to  the 
input  group. 

Since  the  performance  criterion  used  in  this  study 
was  standardized  QPR,  the  predicted  QPR's  were  also  in  stan- 
dard form.   A  conversion  of  these  predicted  ZQPR's  back  to 
raw  QPR's  was  made  using  the  conversion  formula  described 
below. 


PREDICTED  QPR  =  MEAN  QPR  +  SD.QPRX(PRED.ZQPR    ****    PRED.ZQPR) 


SD.  PRED.ZQPR 

2 .   Predictors 

The  original  predictors  were  categorized  into  three 
groups  as  follows: 

1.  Self-reported  biographical  data. 

2.  Graduate  Record  Examination  scores. 

3.  Strong  Vocational  Interest  Blank  Scores. 

A  new  variable  was  derived  denoted  by  INDEX,  using 
self-reported  baccalaureate  QPR  (BQPR)  and  college  quality. 
BQPR  was  extracted  by  using  the  responses  to  questions  15  and 
16  of  the  biographical  questionnaire.   Combinations  of  these 
two  responses  were  converted  to  a  four  point  QPR  scale  in 
the  following  manner: 
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Answer  to  #15        Answer  to  #16       BQPR 


yes 

no 

3.70 

no 

yes 

3.00 

no 

yes 

2.30 

College  Quality  is  a  rating  of  colleges  and  univer- 
sities based  on  mean  Scholastic  Aptitude  Test  scores  [15] . 

C.   RESULT  OF  THE  STUDY 

All  correlation  coefficients  and  prediction  equations 
presented  subsequently  in  this  section  were  determined  from 
the  validation  sample  (N  =  50)  and  all  cross-validation 
correlation  coefficients  were  computed  from  the  hold-out 
sample  (N  =  24) .   Throughout  this  section,  only  the  results 
which  were  significant  at  or  beyond  the  0.10  level  are  reported 
and  only  correlation  coefficients  of  0.25  or  higher  are  re- 
ported.  Appendices  A  and  B  contain  a  complete  list  of  the 
correlation  coefficients  obtained  in  this  study. 

The  heading  "Variable  Entered"  in  the  tables  will  list 
in  order  the  variables  as  they  came  into  the  Stepwise  Multi- 
ple Regression.   The  biographical  questions  were  represented 
by  ITEM  . . ,  thus  ITEM  11  represented  biographical  question 
number  11.   The  SVIB  scales  were  represented  by  VAR  . . ,  as 
listed  in  Appendix  B. 

1.   Stepwise  Multiple  Regression 
a.   Biographical  Data 

Fifty-eight  items  from  the  biographical  question- 
naire were  proposed  to  predict  ZQPR.   ITEM  55,  ITEM  59  and 
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ITEM  60  were  excluded  from  the  investigation  since  even  if 

these  items  entered  into  the  equation,  those  items  would 

have  no  use  as  predictors  (not  available  at  time  of  officer/ 

student  selection).   Those  items  are  listed  as  follows: 

ITEM  55:   Are  you  satisfied  with  your  education 
at  the  Naval  Postgraduate  School? 

ITEM  59:   Do  you  like  your  degree  curriculum? 

ITEM  60:   Would  you  choose  a  different  degree 
curriculum  if  you  could  start  over 
again? 

The  correlation  coefficients  displayed  in  Table  1 

were  calculated  by  the  SPSS  program  using  the  Pearson 

Product-Moment  correlation.   The  table  contains  the  results 

of  correlation  between  ZQPR  and  some  of  the  Biographical 

questions. 

TABLE  I 

CORRELATION  COEFFICIENTS  BETWEEN 
ZQPR  AND  SOME  BIOGRAPHICAL  ITEMS 

BIO  ITEM  #  CORR.  COEFF. 

ITEM  11  0.26 

ITEM  14  -0.26 

ITEM  15  0.34 

ITEM  2  3  -0.32 

ITEM  2  8  -0.2  6 

ITEM  44  0.27 

ITEM  4  8  0.26 

Table  II  contains  the  summary  result  of  the  regres- 
sion analysis  together  with  cross-validation  correlation 
coefficients  determined  by  the  SPSS  program.   Results  only 
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up  to  the  seventh  step  are  reported  in  Table  II.  All 
together  there  were  14  items  accepted  to  be  included  in  the 
equation  at  a  level  of  significance  a  =  0.05,  but  only  ITEM 
15  yielded  a  cross-validation  correlation  greater  than  0.25 
Twelve  percent  of  the  total  variation  was  explained  by  this 
item. 

TABLE  II 
SUMMARY  OF  REGRESSION  OF  ZQPR  ON  BIOGRAPHICAL  ITEMS 


TEP 

VAR. ENTERED 

FVALUE 

MULT . R 

X-VALID 

1 

ITEM  15 

6.329* 

0.34 

0.30 

2 

ITEM  19 

4.740* 

0.44 

-0.04 

3 

ITEM  14 

5.432* 

0.53 

0.04 

4 

ITEM  4  6 

A        O  O  C  * 

0.06 

5 

ITEM  11 

5.977* 

0.65 

0.18 

6 

ITEM  2  3 

5.026* 

0.70 

0.15 

7 

ITEM  2  9 

5.290* 

0.74 

0.16 

Table  III  shows  that  the  correlation  coefficient 
between  ZQPR  and  the  variable  INDEX  was  0.30  and  the  cross- 
validation  correlation  coefficient  was  0.55.   Equation  I 
shows  the  predicted  ZQPR  using  the  variable  INDEX  as  predictor 

TABLE  III 
SUMMARY  OF  REGRESSION  OF  ZQPR  ON  INDEX 

STEP      VAR. ENTERED      FVALUE      MULT.R      X-VALIDATION 
1  INDEX  4.600*      0.30  0.55 
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Equation  I: 

ZQPRP  =  -1.654  +  0.0027  X  INDEX  . 

The  next  step  was  to  investigate  the  relationship 
between  ZQPR,  INDEX  and  the  Biographical  Items. 

ITEM  15  and  ITEM  16  were  excluded  from  this  step 
(together  with  three  items  already  not  in  the  analysis)  since 
these  two  items  had  been  used  to  compute  the  variable  INDEX. 

Table  IV  displays  the  result  of  the  regression  of 
ZQPR  on  INDEX  and  the  remaining  Biographical  Data.   It  can 
be  seen  from  Table  IV  that  only  two  variables  were  signifi- 
cantly accepted  in  the  equation  at  a  <_  0.05.   All  the  seven 
variables  displayed  in  the  table  were  significant  at  a  <  0.10 
Equation  II  displays  predicted  ZQPR  from  the  seven  variables 
selected.   The  cross-validation  correlation  coefficient  for 
this  set  of  predictors  is  0.44. 

TABLE  IV 
SUMMARY  OF  REGRESSION  OF  ZQPR  ON  INDEX  AND  BIO  ITEMS 

STEP      VAR. ENTERED      F VALUE      MULT . R      X -VALIDATION 

1  ITEM  23  5.228*  0.32  0.03 

2  ITEM  11  4.107*  0.42  0.19 

3  ITEM  14  3.926**  0.49  0.24 

4  INDEX  3.840**  0.55  0.39 

5  ITEM  02  6.338**  0.63  0.42 

6  ITEM  56  5.269**  0.68  0.44 

7  ITEM  39  5.083**  0.72  0.44 
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Equation  II: 
ZQPRP  =  -2.89  -  0.610  X  (ITEM  23)  +  0.621  X  (ITEM  11) 

-0.651  X  (ITEM  14)  +  0.003  X  (INDEX)  +  0.934  X  (ITEM  0: 
+0.700  X  (ITEM  56)  +  0.427  X  (ITEM  39)  . 

Not  all  the  items  on  the  Biographical  Questionnaire 
would  be  acceptable  for  use  under  current  personnel  policies; 
some  have  no  face  validity,  or  zero  yes  or  zero  no  answers. 
Further  analysis  was  done  based  on  only  those  items  which 
were  considered  to  be  useful  for  this  study.   These  items 
are  listed  below. 

QUESTION  #  QUESTION 

1.  Did  you  receive  your  commission 
from  the  USNA? 

2.  Did  you  receive  your  commission 
through  an  ROTC  program? 

4.  Is  your  rank  Navy  Lieutenant  or 
below? 

5.  Are  you  a  pilot  or  other  flight 
officer? 

6.  Are  you  a  submarine  officer? 

7.  Are  you  an  unrestricted  line  officer? 

8.  Are  you  a  staff  officer? 

9.  Do  you  have  a  B.S.  (not  a  B.A.) 
degree? 

10.  Have  you  had  at  least  one  year  of 

College  Calculus  at  an  institution 
other  than  the  Naval  Postgraduate 
School? 
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QUESTION  #  QUESTION 

11.  Do  you  speak  at  least  one  language 
other  than  English? 

12.  Do  you  have  a  master's  degree  from 
a  school  other  than  the  Naval 
Postgraduate  School? 

13.  Have  you  taken  any  graduate  courses 
other  than  at  the  Naval  Postgraduate 
School? 

14 .  Have  you  ever  completed  any  courses 
at  night  school  or  through  corres- 
pondence? 

22.  Was  a  branch  of  engineering  your 

undergraduate  major  in  college? 

37.  Did  you  take  any  College-Preparatory 

program  in  high  school? 

44.  Are  you  younger  than  30  years  of  age? 

45.  Wculd  you  expect  to  use  any  skills 
learned  in  graduate  school  in 
subsequent  assignments  in  the  Navy? 

46.  Do  you  expect  to  use  any  graduate 
education  obtained  while  on  active 
duty  in  work  after  you  retire  from 
the  Navy? 

61.  Was  at  least  part  of  your  motivation 

to  remain  in  the  Navy  the  opportunity 
to  receive  postgraduate  education? 


The  following  tables  show  the  summary  results  of  the 
regression  analysis  conducted  using  the  SPSS  program. 

The  inclusion  of  the  first  four  variables  in  the  equa- 
tion was  significant  at  a  <_  0.05.   The  F-test  also  accepted 
the  inclusion  of  seven  variables  at  a  <_   0.10. 

Equations  III-A  and  III-B  describe  the  four  variables 
equation  and  the  seven  variables  equation,  respectively. 
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TABLE  V 

SUMMARY  OF  REGRESSION  OF  ZQPR  ON  INDEX  AND 
SELECTED  BIO  ITEMS 


STEP      VAR. ENTERED  FVALUE  MULT.R      X-VALIDATION 

1  ITEM  14  4.901*  0.31  0.18 

2  INDEX  5.369*  0.43  0.43 

3  ITEM  02  4.431*  0.51  0.48 

4  ITEM  11  4.736*  0.58  0.50 

5  ITEM  12  3.760**  0.62  0.50 

6  ITEM  46  3.631**  0.66  0.50 

7  ITEM  13  3.463**  0.70  0.57 


Equation  III-A: 
ZQPRP  =  -2.079  -  0.748  X  (ITEM14)  +  0.003  X  (INDEX) 
+0.738  X  (ITEM02)  +  0.540  X  (ITEM11)  . 

Equation  III-B: 
ZQPRP  =  -3.751  -  0.884  X  (ITEM  14)  +  0.004  X  (INDEX) 
+0.738  X  (ITEM  02)  +  0.805  X  (ITEM  11) 
+  1.016  X  (ITEM  12)  +  1.402  X  (ITEM  46) 
+  0.554  X  (ITEM  13)  . 

Equation  III-A  yielded  a  0.58  correlation  in  valida- 
tion and  a  0.50  correlation  in  the  cross-validation  sample, 
and  Equation  III-B  yielded  a  0.70  in  validation  and  a  0.57 
in  cross-validation. 
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The  negative  weight  of  ITEM  14  shows  that  s  student 
who  gave  a  "no"  answer  to  this  item  had  a  better  academic 
performance  than  a  student  who  gave  a  "yes".   However,  even 
if  this  item  happens  to  be  a  good  predictor  for  academic 
performance,  this  information  might  not  be  available  on  an 
officer's  record.   Thus  further  analysis  was  performed  by 
excluding  ITEM  14.   Table  VI  displays  the  summary  results 
of  this  analysis. 

TABLE  VI 

SUMMARY  OF  REGRESSION  OF  ZQPR  ON  INDEX  AND 
SELECTED  BIO  ITEMS  (EXCLUDING  ITEM  14) 

STEP      VAR. ENTERED      FVALUE      MULT.R      X- VALIDATION 

1  INDEX  4.602*  0.30  0.55 

2  ITEM  02  3.905**  0 . 3S  0.56 

3  ITEM  04  3.236**  0.45  0.52 

4  ITEM  11  3.650**  0.51  0.51 

5  ITEM  12  3.843**  0.57  0.51 

Equation  IV: 
ZQPR  =  -2.369  +  0.003  X  (INDEX)  +  0.761  X  (ITEM  02) 
+  0.551  X  (ITEM  04)  +  0.549  X  (ITEM  11) 
+  1.630  X  (ITEM  12) . 

Only  INDEX  was  accepted  to  be  included  in  the  equation 
at  a  £  0.05.   Five  variables  were  significant  at  a  <_   0.10. 
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Equation  IV  shows  predicted  ZQPR  with  five  varia- 
bles.  The  validation  correlation  of  Equation  IV  was  0.59 
and  the  cross-validation  between  predicted  ZQPR  and  actual 
ZQPR  for  Equation  IV  was  0.51. 

b.   Graduate  Record  Examination 

A  total  of  498  NPS  students  took  the  Graduate 
Record  Exam,  including  80  students  registered  in  the  Operations 
Research/Systems  Analysis  curriculum. 

The  following  table  shows  some  descriptive 
statistics  for  the  GRE  data. 

TABLE  VII 
DESCRIPTIVE  STATISTICS  FOR  THE  GRE  DATA 

VERBAL 


STATISTICS 

yurui  j.  . 

L    J.  £  I  -L    J.    VJJ 

N   =    498 

N   =    80 

N   =    498 

N   =    80 

MEAN 

660.02 

711.74 

532.10 

549.62 

MEDIAN 

680.00 

715.00 

530.00 

540.00 

VARIANCE 

6500.00 

3569.00 

9949.00 

9596.00 

S.D. 

80.60 

59.70 

99.70    • 

97.90 

RANGE 

440.00 

290.00 

590.00 

480.00 

MAX 

820.00 

820.00 

830.00 

790.00 

MIN 

380.00 

530.00 

240.00 

310.00 

Table  VIII  contains  the  correlation  coefficients 
between  ZQPR  and  GRE-Quantitative ,  GRE-Verbal  and  GRE-Total. 
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TABLE  VIII 

CORRELATION  BETWEEN  ZQPR  AND  GRE 
(N  =  50) 


GRE 

QUANTITATIVE 

VERBAL 

TOTAL  (QUANT  +  VERB) 


CORRELATION  COEFFICIENT 

0.45 
0.41 
0.49 


Table  IX  contains  the  summary  results  of  a 
regression  of  ZQPR  on  GRE  scores. 

Equation  V  shows  the  constant  and  variable  weights 
for  this  predictor  system.   The  cross-validation  correlation 
coefficient  of  the  predicted  ZQPR  was  0.69. 

TABLE  IX 

SUMMARY  FROM  REGRESSION  OF  ZQPR  ON  GRE  SCORES 

(N  =  50) 


STEP      VAR. ENTERED 

1  QUANT 

2  VERBAL 


FVALUE 

12.034* 
3.376** 


MULT . R 

0.45 
0.50 


X-VALIDATION 

0.55 
0.69 


Equation  V: 


ZQPR   =    -5.271    +    0.054    X    (GRE    QUANT)    +    0.026    X     (GRE    VERB) 
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c.   SVIB  Scales 

Table  X  contains  the  correlation  coefficients 
of  ZQPR  with  SVIB  scales  which  are  >_  0.25. 

TABLE  X 

CORRELATION  COEFFICIENTS  BETWEEN 
ZQPR  AND  SOME  SVIB  SCALES 
(N  =  74) 


SVIB  SCALE 

PHYSICAL  THERAPIST 

VETERINARIAN 

PSYCHOLOGIST 

MATHEMATICIAN 

CHEMIST 

FOREST  SERVICE 

POLICEMAN 

YMCA  SECRETARY 

RECREATION  ADMINISTRATOR 

CPA  OWNER 

SENIOR  CPA 

MORTICIAN 

COMPUTER  PROGRAMMER 

ACADEMIC  ACHIEVEMENT 

SPECIALIZATION  LEVEL 


CORRELATION  COEFFICIENT 

-0.27 
-0.37 

0.36 

0.39 

0.26 
-0.33 
-0.31 
-0.36 
-0.26 

0.31 

0.33 

0.31 

0.37 

0.32 

0.32 


The  summary  results  of  the  regression  of  ZQPR 
on  the  SVIB  scales  is  shown  in  Table  XI .   It  can  be  seen 
that  the  four  predictor  system  was  significant  at  a  <_  0.05 
and  the  six  predictor  system  was  significant  at  a  _<  0.10. 

The  equation  for  four  predictor  systems  and  six 
predictor  system  are  described  by  Equation  VI-A  and  VI-B, 
respectively.   Equation  VI-A  yielded  a  0.67  correlation 
coefficient  in  the  validation  sample  and  a  cross-validation 
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correlation  of  0.52.   Equation  VI-B  yielded  a  0.73  correlation 
in  validation  sample  and  a  cross-validation  correlation  of 
0.59. 

TABLE  XI 
SUMMARY  OF  REGRESSION  OF  ZQPR  ON  SVIB  SCALES 

STEP      VAR. ENTERED      FVALUE      MULT.R      X-VALIDATION 

1  VAR  19  8.379*  0.39  0.20 

2  VAR  55  8.988*  0.54  0.37 

3  VAR  27  4.806*  0.60  0.45 

4  VAR  14  7.490*  0.67  0.52 

5  VAR  26  3.058**  0.70  0.57 

6  VAR  60  3.418**  0.73  0.59 

Ecruaticn  VI— As 
ZQPR  =  2.234  -  0.023  X  (VAR  19)  +  0.042  X  (VAR  55) 
-0.057  X  (VAR  27)  -  0.060  X  (VAR  14) 

Equation  VI-B 
ZQPR  =  1.059  -  0.022  X  (VAR  19)  +  0.025  X  (VAR  55) 
-  0.093  X  (VAR  27)  -  0.070  X  (VAR  14) 
+  0.047  X  (VAR  26)  +  0.034  X  (VAR  60). 

d.   GRE  and  Selected  Biographical  Items 

The  combination  of  selected  items  from  the 
biographical  inventory  and  the  Graduate  Record  Exam  was 
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analyzed  to  see  whether  this  combination  could  provide  any 
information  about  academic  performance.   The  following  table 
shows  the  summary  results  from  the  regression  of  ZQPR  on  GRE 
and  selected  biographical  items.   Equation  VII  predicts  ZQPR 
with  five  predictors.   These  five  variables  are  significant 
at  a  <  0.05.   The  correlation  coefficient  was  0.72  and  the 
cross-validation  correlation  coefficient  was  0.64. 

TABLE  XII 

SUMMARY  OF  REGRESSION  OF  ZQPR  ON  GRE 
AND  SELECTED  BIO  ITEMS 


X-VALIDATION 

0.69 
0.60 
0.61 
0.61 
0.64 


Equation  VII : 
ZQPRP  =  -5.409  +  0.032  X  (GRE  TOTAL)  -  0.788  X  (ITEM  14) 
+  0.514  X  (ITEM  11)  +  0.736  X  (ITEM  02) 
+  0.002  X  (INDEX) . 

As  can  be  seen  from  Equation  VII,  ITEM  14  was 
again  included  in  the  equation  with  a  negative  weight.   Re- 
moving ITEM  14  from  the  set  of  candidates  for  predictors 


TEP 

VAR. ENTERED 

FVALUE 

MULT . R 

1 

13.977* 

O  A  Q 

2 

ITEM  14 

7.911* 

0.59 

3 

ITEM  11 

4.638* 

0.64 

4 

ITEM  02 

4.627* 

0.68 

5 

INDEX 

4.537* 

0.72 
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yielded  the  results  as  shown  in  Table  XIII  and  Equation  VIII 
Only  three  predictors  were  significant.   The  correlations  in 
validation  and  in  cross-validation  were  0.60  and  0.66 
respectively. 

TABLE  XIII 

SUMMARY  OF  REGRESSION  OF  ZQPR  ON  GRE 
AND  SELECTED  BIO  ITEMS 
(EXCLUDING  ITEM  14) 


TEP 

VAR. ENTERED 

FVALUE 

MULT . R 

X-VALI 

1 

GRE  TOTAL 

13.977* 

0.48 

0.69 

2 

ITEM  4  4 

4.732* 

0.55 

0.64 

3 

ITEM  13 

3.772* 

0.60 

0.66 

Equation  VIII: 
ZQPR  =  -4.322  +  0.031  X  (GRE  TOTAL)  +  0.653  X  (ITEM  44) 
+  0.593  X  (ITEM  13) . 

Table  XIV,  together  with  Equation  IX ,  displays 
the  results  if  INDEX  was  forced  into  the  equation. 
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TABLE  XIV 

SUMMARY  OF  REGRESSION  OF  ZQPR  ON  GRE 
SELECTED  BIO  ITEMS  (EXCLUDING  ITEM  14) 
AND  INDEX  (FORCED) 


STEP  VAR. ENTERED  FVALUE  MULT.R  X-VALIDATION 

1  INDEX  4.602*  0.30        0.55 

2  GRE  TOTAL  10.596*  0.51        0.72 

3  ITEM  04          3.495**  0.56        0.68 

4  ITEM  02  4.981**  0.62        0.66 

5  ITEM  11  4.196**  0.66        0.66 


Equation  IX: 
ZQPRP  =  -5.652  +  0.0015  X  (INDEX)  +  0.031  X  (GRE  TOTAL) 
+  0.637  X  (ITEM  04)  +  0.720  X  (ITEM  02) 
+  0.471  X  (ITEM  11) . 

e.   GRE  and  SVIB 

The  following  table  (Table  XV)  presents  the 
results  of  the  step-wise  regression  for  the  combination  of 
GRE  scores  and  SVIB  scales.   It  can  be  seen  from  the  table 
that  the  multiple  correlation  coefficient  attained  with  four 
predictors  was  0.69;  yielded  a  cross-validation  correlation 
coefficient  of  0.64.   Equation  X  shows  the  weight  for  each 
predictor. 
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TABLE  XV 
SUMMARY  OF  REGRESSION  OF  ZQPR  ON  SVIB-GRE 

STEP      VAR. ENTERED      FVALUE      MULT.R      X-VALIDATION 

1  GRE  TOTAL  13.977*  0.48  0.69 

2  VAR  38  5.150*  0.55  0.78 

3  VAR  11  6.988*  0.63  0.79 

4  VAR  41  6.2.69*  0.69  0.64 

5  VAR  40  3.839**  0.72  0.61 

Equation  X: 
ZQPRP  =  -5.586  +  0.020  X  (GRE  TOTAL)  +  0.010  X  (VAR  38) 
+  0.064  X  (VAR  11)  +  0.041  X  (VAR  41). 

As  can  be  seen,  INDEX  was  not  strong  enough  to 
be  included  in  the  equation.   Forcing  INDEX  into  the  equation 
yielded  the  results  shown  in  Table  XVI  and  Equation  XI. 

TABLE  XVI 

SUMMARY    OF    REGRESSION    OF    ZQPR   ON    SVIB,    GRE 
AND    INDEX     (FORCED) 

STEP  VAR. ENTERED  FVALUE  MULT.R  X-VALIDATION 

1  INDEX  4.602*  0.30  0.55 

2  GRE    TOTAL  10.596*  0.51  0.72 

3  VAR   55  5.160*  0.60  0.73 

4  VAR   19  5.458*  0.64  0.64 
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Equation  XI : 
ZQPRP  =  -4.234  +  0.002  X  (INDEX)  +  0.019  X  (GRE  TOTAL) 
+  0.028  X  (VAR  55)  -  0.021  X  (VAR  19)  . 

f.   Selected  BIO  -  SVIB 

The  following  table  presents  the  results  of  the 
stepwise  regression  for  the  combination  of  selected  bio- 
graphical items  and  SVIB  scales,  including  INDEX.   Equation 
XII  shows  the  weight  for  each  predictor.   The  correlation 
attained  was  0.77,  and  the  cross-validation  was  0.50 

TABLE  XVII 

SUMMARY  OF  REGRESSION  OF  ZQPR  ON  SVIB 
SCALES  AND  SELECTED  BIO  ITEMS 

STEP      VAR. ENTERED      FVALUE      MULT.R      X- VALIDATION 

1  VAR  19  8.379*  0.39  0.20 

2  VAR  55  8.988*  0.54  0.37 

3  VAR  27  4.806*  0.60  0.45 

4  VAR  14  7.490*  0.67  0.52 

5  ITEM  44  5.878*  0.72  0.51 

6  ITEM  12  8.593*  0.77  0.50 

Equation  XII: 

ZQPRP  =  2.185  -  0.020  X  (VAR  19)  +  0.040  X  (VAR  55) 

-  0.062  X  (VAR  27)  -  0.063  X  (VAR  14) 

+  0.563  X  (ITEM  44)  +  1.897  X  (ITEM  12). 
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It  can  be  seen  that  INDEX  was  not  included  in 
Equation  XII.   Forcing  INDEX  into  the  equation  yielded  the 
following  results: 

TABLE  XVIII 

SUMMARY  OF  REGFESSION  OF  ZQPR  ON 
SVIB  -  BIO  -  INDEX  (FORCED) 


TEP 

VAR. ENTERED 

FVALUE 

MULT . R 

X-VALID 

1 

INDEX 

4.602* 

0.30 

0.55 

2 

VAR  55 

7.895* 

0.47 

0.64 

3 

VAR  19 

8.951* 

0.59 

0.53 

4 

ITEM  12 

3.939** 

0.64 

0.52 

5 

VAR  2  7 

4.477** 

0.68 

0.57 

6 

VAR  14 

8.307** 

0.74 

0.61 

7 

ITEM  44 

5.498** 

0.78 

0.54 

Equation  XIII: 

ZQPRP  =  1.690  +  0.001  X  (INDEX)  +  0.040  X  (VAR  55) 

( 

-  0.019  X  (VAR  19)  +  1.852  X  (ITEM  12) 

-  0.061  X  (VAR  27)  -  0.062  X  (VAR  14) 
+  0.052  X  (ITEM  44)  . 

g.   All  Variables 

Table  XIX,  together  with  Equation  SIV,  displays 
the  results  of  regressing  ZQPR  on  all  candidates  for  predic- 
tors.  The  multiple  correlation  attained  with  6  predictors 
was  0.74,  and  yielded  a  cross-validation  correlation 
coefficient  of  0.60. 
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TABLE  XIX 

SUMMARY  OF  REGRESSION  OF  ZQPR 
ON  ALL  VARIABLES 


STEP  VAR. ENTERED  FVALUE  MULT.R  X- VALIDATION 

1  GRE  TOTAL  13.977*  0.48  0.69 

2  VAR  38  5.150*  0.55  0.78 

3  VAR  11  6.988*  0.63  0.79 

4  VAR  41  6.269*  0.69  0.64 

5  ITEM  04          3.986**  0.72  0.60 

6  ITEM  22          3.044**  0.74  0.60 


Equation  XIV: 
ZQPRP  =  -6.349  +  0.019  X  (GRE  TOTAL)  +  0.008  X  (VAR  38) 
+  0.071  X  (VAR  11)  +  0.050  X  (VAR  41) 
+  0.54  8  X  (ITEM  0  4)  +  0.375  X  (ITEM  22)  . 

Forcing  INDEX  into  the  equation  yielded  the 
results  presented  in  Table  XX  and  Equation  XV.   The  multiple 
correlation  coefficient  attained  with  seven  predictors  was 
0.78  and  the  cross-validation  correlation  was  0.70. 
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TABLE  XX 

SUMMARY  OF  REGRESSION  OF  ZQPR  ON 
ALL  VARIABLES  WITH  INDEX  FORCED 
INTO  THE  EQUATION 


STEP  VAR. ENTERED  FVALUE  MULT.R      X-VALIDATION 

1  INDEX  4.602*  0.30  0.55 

2  GRE  TOTAL  10.596*  0.51  0.72 

3  ITEM  14  8.250*  0.61  0.53 

4  VAR  38  7.311*  0.72  0.70 

5  ITEM  02  5.588*  0.73  0.70 

6  VAR  07  4.170*  0.75  0.60 

7  ITEM  13  4.939*  0.78  0.70 


Equation  SV: 
ZQPRP  =  -6.773  +  0.002  X  (INDEX)  +  0.027  X  (GRE  TOTAL) 
-  0.69  3  X  (ITEM  14)  +  0.037  X  (VAR  38) 
+0.600  X  (ITEM  02)  +  0.024  X  (VAR  07) 
+  0.532  X  (ITEM  13)  . 

2 .   Summary  Results  from  the  Stepwise  Regression  Analysis 
Stepwise  regression  using  biographical  information, 
the  Strong  vocational  interest  Blank  and  the  Graduate  Record 
Examination  yielded  several  prediction  equations  which  produced 
significant  results  in  cross-validation. 

Four  of  these  performance  prediction  equations  were 
selected  for  further  discussion  in  the  subsequent  paragraphs. 
Their  selection  was  based  on  the  number  of  predictors  involved 
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and  on  the  correlation  coefficients  attained  in  the  validation 
and  cross-validation  samples.   These  four  systems  of  predictors 
are: 

a.  A  combination  of  GRE  and  SVIB,  consisting  of 
four  predictors,  all  of  which  were  significant 
at  a  <  0.05. 

b.  A  combination  of  GRE  and  BIO  items,  consisting 

of  five  predictors,  all  of  which  were  significant 
at  a  £  0.05. 

c.  A  combination  of  SVIB  and  BIO  items,  consisting 
of  six  predictors,  all  of  which  were  significant 
at  a  <   0.05. 

d.  A  combination  of  GRE,  SVIB  and  BIO  items,  con- 
sisting of  seven  predictors,  all  of  which  were 
significant  at  a  <_  0.05. 

Each  of  these  systems  of  predictors  for  academic 
performance  is  briefly  discussed  in  the  following  paragraphs. 

a.   A  Combination  of  GRE  and  SVIB  Scores 

The  set  of  predictors  for  academic  performance 
composed  of  a  combination  of  GRE  scores  and  SVIB  scores  pro- 
duced a  0.69  validation  correlation  and  a  0.64  cross-validation 
correlation.   Table  XXI  displays  the  constant,  the  variables 
and  the  weights  for  the  prediction  equation  described  by 
Equation  X. 
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TABLE  XXI 
COMBINATION  OF  GRE  -  SVIB 


VARIABLES 


DESCRIPTION 


WEIGHT 


GRE  TOTAL 
VAR  3  8 
VAR  11 
VAR  41 

CONSTANT 


GRE  VERBAL  +  GRE  QUANT 
CPA  OWNER 
MATHEMATICIAN 
CREDIT  MANAGER 


+0.020 
+0.010 
+0.064 
+0.041 
-5.586 


The  formula  for  converting  predicted  ZQPR  to 
raw  QPR  is  provided  by  Equation  XVI. 
Equation  XVI : 

PRED.  RAW  QPR  -  0.4  56  X  (PRED.  ZQPR)  +  3.30  . 

b.   A  Combination  of  GRE  Scores  and  BIO  Data 

The  set  of  predictors  for  academic  performance 
composed  of  a  combination  of  GRE  scores  and  of  BIO  items 
produced  a  0.72  validation  correlation  and  a  0.64  cross- 
validation  correlation.   Table  XXII  displays  the  constant, 
the  variables  and  the  weights  for  the  prediction  equation 
described  by  Equation  VII. 
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TABLE  XXII 
COMBINATION  OF  GRE  -  BIO 


VARIABLES 


DESCRIPTION 


WEIGHT 


GRE  TOTAL 
ITEM  14 

ITEM  11 


ITEM  02 

INDEX 

CONSTANT 


GRE  VERBAL  +  GRE  QUANT 
COMPLETED  NIGHT  SCHOOL 

(OR  THROUGH  CORRESP) 
SPEAK  AT  LEAST  ONE 

LANGUAGE  OTHER  THAN 

ENGLISH 
ROTC  GRADUATE 
BQPR  X  COLL.  QUALITY 


+  0.032 
-  0.788 


+  0.514 
+  0.736 
+  0.002 
-  5.409 


The  formula  for  converting  predicted  ZQPR  to 
raw  QPR  is  provided  by  Equation  XVII. 
Equation  XVII: 

PRED.  RAW  QPR  =  0.337  X  (PRED.  ZQPR)  +  3.41 

c.   A  combination  of  SVIB  Scores  and  BIO  Data 

The  set  of  predictors  for  academic  performance 
composed  of  a  combination  of  SVIB  scores  and  BIO  items  pro- 
duced a  0.77  validation  correlation  and  a  0.50  cross-valida- 
tion correlation.   Table  XXIII  displays  the  constant,  the 
variables  and  the  weights  for  the  prediction  equation 
described  by  Equation  XII. 
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TABLE  XXIII 
COMBINATION  OF  SVIB  -  BIO 


VARIABLES 

VAR  19 
VAR  55 
VAR  2  7 
VAR  14 

ITEM  4  4 
ITEM  12 

CONSTANT 


DESCRIPTION 

WEIGHT 

FOREST    SERVICE 

-    0.020 

COMPUTER   PROGRAMMER 

+    0.039 

YMCA    SECRETARY 

-    0.062 

ENGINEER 

-    0.063 

YOUNGER   THAN    30    YEARS 

+    0.563 

POSSESSING   MASTER'S 

DEGREE 

+    1.897 

+    2.185 

The  formula  for  converting  predicted  ZQPR  to 
raw  QPR  is  provided  by  Equation  XVIII. 
Equation  XVIII: 

PRED.  RAW  QPR  =  0.346  X  (PRED.  ZQPR)  +3.38 

d.   A  combination  of  GRE ,  SVIB  and  BIO  Data 

The  set  of  predictors  for  academic  performance 
composed  of  a  combination  of  GRE  scores,  SVIB  scores  and 
BIO  items  produced  a  0.78  validation  correlation  and  a  0.70 
cross-validation  correlation.   Table  XXIV  displays  the 
constants,  the  variables  and  the  weights  for  the  prediction 
equation  described  by  Equation  XV. 
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TABLE  XXIV 
COMBINATION  OF  GRE"  -  SVIB  -  BIO 


VARIABLES 

INDEX 
GRE  TOTAL 
ITEM  14 

VAR  38 
ITEM  02 
VAR  0  7 
ITEM  13 


CONSTANT 


DESCRIPTION 

BQPR  X  COLL. QUALITY 
GRE  VERBAL  +  GRE  QUANT 
COMPLETED  NIGHT  SCHOOL 

OR  THROUGH  CORRESP. 
CPA  OWNER 
ROTC  GRADUATE 
PHYSICIAN 
HAVE  TAKEN  GRADUATE 

COURSE  OTHER  THAN  AT 

NPS 


WEIGHT 

+  0.002 
+  0.027 

-  0.693 
+  0.037 
+  0.598 
+  0.024 


+  0.532 
-  6.773 


The  formula  for  converting  predicted  ZQPR  to 
raw  QPR  is  provided  by  Equation  XVIX. 
Equation  XVIX: 

PRED.  RAW  QPR  =  0.318  X  (PRED.  ZQPR)  +3.39 

3.   Jackknif ing 

The  four  equations  that  had  been  selected  were  further 
studied  to  obtain  the  regression  coefficients  using  the  Jack- 
knife  procedure.   The  regression  coefficients  obtained  from 
this  procedure  were  used  to  compute  the  correlation  coeffi- 
cient and  were  then  compared  with  the  results  obtained  from 
the  Stepwise  Multiple  Regression.   The  procedure  for  Jack- 
knifing  is  presented  in  the  previous  chapter.   In  this  study, 
the  validation  sample  was  divided  into  ten  subsamples,  each 
with  five  cases. 
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The  following  table  presents  the  comparisons  between 
the  regression  coefficients  obtained  from  using  Stepwise 
Multiple  Regression  and  the  Jackknife  procedure. 

It  can  be  seen  from  the  table  that  there  are  no 
significant  differences  between  estimators  obtained  from 
the  Stepwise  Multiple  Regression  and  the  Jackknife  procedure. 
We  are  on  firmer  ground  in  concluding  that  the  variable's 
weights  in  this  study  are  unviased  and  in  concluding  that 
it  is  no  longer  necessary  to  assume  that  the  error  is  normally 
distributed. 

4 .   Expectancy  Charts 

In  order  to  more  easily  visualize  the  results  from 
this  study,  expectancy  charts  were  derived,  using  the  predic- 
tion equations  obtained  from  the  Jackknife  method  and  the 
conversion  formula. 

An  expectancy  chart  is  defined  as  a  table  of  numbers 
from  which  one  may  determine  the  likelihood  or  probability 
that  a  particular  individual  or  group  of  individuals  will  at- 
tain a  specified  definition  of  "superiority"  [16]  .   There  are 
two  types  of  expectancy  charts:   institutional  and  individual. 
The  institutional  expectancy  chart  indicates  what  will  happen, 
institutionally,  if  a  particular  minimum  cutting  score  is 
applied.   For  example,  a  minimum  cutting  score  of  3.60  will 
yield  87%  who  are  "superior".   The  individual  expectancy 
chart  is  designed  to  determine  the  likelihood  or  probability 
that  a  specific  individual  will  achieve  "superiority",  given 
his  score. 
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TABLE  XXV 

COMPARISON  BETWEEN  ESTIMATORS  OBTAINED  FROM  THE  STEPWISE 
MULTIPLE  REGRESSION  (SMR)  AND  THE  JACKKNIFE  PROCEDURE 


VARIABLES 


SMR 


WEIGHT 

JACKKNIFE 


Combination  of  GRE  -  SVIB 

GRE  TOTAL 
VAR  38 
VAR  11 
VAR  41 

CONSTANT 

VALIDATION  CORR.  COEFF. 

X-VALIDATION 


+0.02034 
+0.01007 
06399 
04053 
58609 
68908 
6435 


+0 

+  0 

-5 

0 

0 


+0.023277 
+0.009545 
+0.059148 
+0.03623 
-5.68205 
0.7006 
0.6613 


b.   Combination  of  GRE  -  SVIB 

GRE  TOTAL 
ITEM  14 
ITEM  11 
ITEM  02 
INDEX 

VALIDATION  CORR.  COEFF. 
X-VALIDATION  CORR.  COEFF, 


+  0 
-0 
+0 
+0 
+0 


03169 
78837 
51393 
73564 
00224 


->  .  _»  \j  zi  <L  •* 

0.71554 
0.6368 


+0.031186 
-0.031186 
504795 
802268 
002285 
401033 


+  0 
+  0 
+  0 
-5 


0.7147 
0.6385 


Combination  of  SVIB  -  BIO 

VAR  19  -0.01989 

VAR  55  +0.03945 

VAR  27  -0.06236 

VAR  14  -0.06290 

ITEM  44  +0.56286 

ITEM  12  +1.89703 

CONSTANT  +2.18459 

VALIDATION  CORR.  COEFF.  0.774  39 

X-VALIDATION  CORR.  COEFF.  0.5066 


-0 
+  0 
-0 
-0 
+0 

+1 
+1 

0 


0224 

03578 

05398 

04654 

55097 

8484 

7253 

7773 


0.5091 


d.   Combination  of  SVIB-BIO-GRE 

INDEX 

GRE  TOTAL 

ITEM  14 

VAR  3  8 

ITEM  02 

VAR  07 

ITEM  13 

CONSTANT 

VALIDATION  CORR.  COEFF. 

X-VALIDATION  CORR.  COEFF 


+0.00231 
+0.02736 
-0.69271 
+0.03732 
+0.59807 
+0.02425 
+0.53236 
-6.77349 
0.78393 
0.7021 


+0.002301 
+0.02654 
68273 
038643 
655238 
02627 
57942 
79821 
7831 


-0 
+0 
+  0 
+  0 
+  0 
-6 
0 


0.6976 
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"Superiority"  is  defined  as  the  level  of  academic 
performance  of  a  student  who  had  a  QPR  greater  than  or  equal 
to  a  median  QPR  of  the  74  subjects  in  this  study. 
5 .   Pattern  Analysis 

Pattern  analyses  were  conducted  using  Biographical 
Questionnaire  data  from  the  validation  group.  The  results 
then  were  cross-validated  on  the  hold-out  group. 

The  first  item  selected  was  ITEM  17  and  yielded  a 
0.27  correlation  coefficient  with  the  criterion  score:   the 
second  item  selected  was  ITEM  12  and  together  with  ITEM  17 
yielded  a  0.44  correlation  with  the  criterion  score.   The 
third  item  selected  was  ITEM  5.   The  three  items  selected 
produced  a  0.53  correlation  coefficient.   The  cross-validation 
correlation  coefficient  for  the  three  items  selected  was 
-0.014,  which  is  not  significant.   The  reduction  in  correla- 
tion was  probably  due  to  the  relatively  small  samples  in 
the  validation  and  hold-out  samples.   There  were  no  subjects 
who  had  the  yes  -  yes  -  no  pattern  in  the  validation  group, 
hence  a  pattern  score  could  not  be  computed,  and  a  zero 
pattern  score  was  assigned  to  that  pattern,  while  in  the 
cross-validation  sample  there  were  two  subjects  who  had  that 
pattern.   This  caused  the  correlation  coefficient  to  drop 
significantly  in  the  validation  sample. 


56 


EXPECTANCY  CHARTS 


GROUP 

!MIN   SCORE 

1              PERCENTAGE  THAT  WILL  BE  SUPERIOR 

3.62 

20% 

i  umiminiuiimiiiiiiimimiiim 

80% 

3M 

40% 

\iiuiiiiiiiiiiiiiiuiinmiiiuiniiii 

80% 

3.20 

60fo 

imiiiiuiiiiiiiiiuunnniiiii 

e% 

3.02 

80% 

\iuiiimiiiiiimmmiiiii 

6<yfo 

2.68 

ALL       ! 

iiiiiiiimumimiiin 

52% 

Fig.  1  -  a.  Institutional  expectancy  chart  based  on  combination  of 
GRE  -  SVIB. 
Conversion  formula:  Raw  QPRP  =  0.5022  X  (ZQPRP)  +  3.32 


TABLE  SCORE 

CHANCES   IN  A  HUNDRED  OF  BEING  SUPERIOR 

3.64—  4.00         < 

\mmmimmmmmmimmm 

80 

3.41  -  3.61 

\iiimiumnmimiimiuiiiiiii! 

80 

3.20  -  3.40 

\niiii)i)iii)ii))ii)iiu 

47 

3.02  -  3.19 

Mini  mi  mini 

33 

2.68  -  3.01 

! ////////// 

21% 

Fig.   1  -  b.   Individual  expectancy  chart  based  on  combination  of 
GRE  -  SVIB. 
Conversion  formula:   Raw  QPRP  -  0.5022  X  (ZQPRP)  +  3.32 
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■GROUP 

IMIN    SCORE 

:              PERCENTAGE  THAT  WILL  BE  SUPERIOR                                  J 

3.63 

93*     ! 

20% 

iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiimi 

:     3.39 

QJ%    ! 

kO% 

\imiiuiunuuiiiiiiiiiiiiiiimiiiiii 

3.20 

71%     ! 

60%       ! 

innniiiiiiimiiiiiiiimiiimi 

3. 02 

59%    l 

SO%       ! 

\iiniiiiiiiiiiiiiiiiiiiiiiiii 

2.19      ! 

Wo  i 

ALL       ! 

//////////////////////// 

Fig.  2  -  a.   Institutional  expectancy  chart  based  on  combination  of 
GRE  -  BIO. 
Conversion  formula:  Raw  QPRP  =  0.^917  X  (ZOJPRP)  +3.3^ 


!       TABLE  SCORE 

CHANCES   IN  A  HUNDRED  OF  BEING  SUPERIOR                          ! 

!      3.63  -  JJ-.oo 

liiiuiniiiiiuiiiiiiuiuuinniiuuiuiu  93    .• 

!       3.39  -  3.62 

////////////////////////////////////                         73       ! 

!       3.20  -  3.38 

iiiiiiimimniiiiiiii                       47    ! 

•       3.02  -  3.19 

ill!  Ml  III                                                     21     i 

!       2.19  -  3.01 

;ZZZ7                                                        7     ; 

Fig.  2  -  b.   Individual  expectancy  chart  based  on  combination  of 
GRE  -  BIO. 
Conversion  formula:  Raw  QPRP  -  0.^917  X  (ZQPRP)  +3-3^ 
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GROUP 

WIN    SCORE 

1                   PERCENTAGE  THAT  WILL  BE  SUPERIOR 

!     3.66 

! 

ZQffo        , 

\iiimiiimmmmmiiiiiiiiiiiiimiiii 

87% 

3.37 

Wo       1 

\iiimiimmiiiniiiiimiiiiniiiii 

11% 

3.21 

6($       ! 

Jiiiiiiiiiiinimiiiiiniiiini 

(m 

3.06 

80?o 

jiiiiiimniniinnnnmi 

60% 

2,55      : 

ALL 

Jiiiinniiiiiiiiiiiinii 

50% 

Fig.   3  -  a.  Institutional  expectancy  chart  based  on  combination  of 
SVIB  -  BIO. 
Conversion  formula:  Raw  QPRP  «  0,4541  X  (ZQPRP)  +3.31. 


TABLE  SCORE 

CHANCES   IN  A  HUNDRED  OF  BEING  SUPERIOR 

3.66  -  4.00          ! 

\iuiuniniiiniiiiiiiiiiiimmniuiiin 

87 

3.37  -  3.65 

Mimiiimmiimiiimimim 

67 

3.21  -  3.36 

Miumummiimui 

46 

3.06  -  3.20 

\iiimmnmmm 

40 

2.55  -  3.06 

;ZZZ7 

7 

Fig.   3  -  b.   Individual  expectancy  chart  based  on  combination  of 
SVIB  -  BIO. 
Conversion  formula:  Raw  QPRP  =  0.4541  X  (ZQPRP )  +  3.31 
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GROUP 

!min  score 

:                   PERCENTAGE  THAT  WILL  BE  SUPERIOR 

3.64 

20% 

Miiniiiimnniuiiuiiiiuniiiiiuiiiiuii 

93% 

3.^3 

40% 

\imiiimiiiiiiiiiuuinnnnmmiii 

m 

3.20 

GQffo       ! 

uiiumniimimmmmim 

67% 

3.01 

80%       ! 

umiimmmiiimnimi 

59fo 

2.31 

ALL       ! 

iiiiiiiiiiiiiiiimnim 

50fo 

Fig.  4  -  a.   Institutional  expectancy  chart  based  on  combination  of 
SVIB  -  BIO  -  GRE". 
Conversion  formula:  Raw  QPRP  =  0.3826  X  ( ZQPRP )  +  3.35 


TABLE  SCORE 


3.64  -  4.00 


1   3.^3  -  3.63 


3.20  -  3.^2 


CHANCES  IN  A  HUNDRED  OF  BEING  SUPERIOR 


ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ7  9* 


ZZZZZZZZZZZHIZZZZZZZZZZZZZZZZZZZZZ7 


ZZZZZZZZZZZZZZZZZZZ7 


1% 


40% 


3.01  -  3.19 


ZZZZZZZZZZZZZZ7 


2?% 


2.31  -  3.oo 


ZZZZZZ7 


14% 


Fig.  4  -  b.  Individual  expectancy  chart  based  on  combination  of 
SVIB  -  BIO  -  GRE. 
Conversion  formula:  Raw  QPRP  -  0.3826  X  (ZQPRP)  +  3.35 
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V.   CONCLUSION  AND  RECOMMENDATION 

A.   CONCLUSION 

1.  Four  prediction  equations  for  predicting  academic 
performance  have  been  developed  in  this  study,  using  Bio- 
graphical Questionnaire  (including  prior  academic  perfor- 
mance) ,  Strong  Vocational  Interest  Blank  and  Graduate  Record 
Examination  data. 

2.  Data  from  these  three  instruments,  as  well  as  data 
from  any  combination  of  two  (or  three)  of  them  provided  a 
higher  potential  for  predicting  academic  performance  than 
from  using  prior  academic  performance,  as  measured  by  self- 
reported  u11clsrcr2r3.cU1s1.te  QPR  2nd  College  Qualitv. 

3.  The  74  U.S.  Navy  officers  in  the  sample  group  had 
already  been  subjected  to  screening  processes,  hence  they 
were  a  more  homogeneous  group  than  U.S.  Navy  officers  in 
general.   This  created  a  complexity  that  one  cannot  explore 
fully.   For  example,  the  Educational  Potential  Classification 
requires  an  over-all  average  grades  at  least  2.50  for  direct 
input  into  a  technical  curriculum.   This  requirement  reduced 
the  variability  of  Baccalaureate  QPR  within  the  group  and 
hence  the  INDEX.   Under  these  circumstances  the  correlation 
of  INDEX  with  the  criterion  score  was  reduced.   Its  true 
value  would  not  have  been  changed,  but  the  evidence  available 
would  not  have  permitted  to  see  that  value.   Those  who  would 


61 


have  failed  because  of  the  low  prior  academic  performance 
would  have  been  cut  off  at  the  source  [3] . 

B.   RECOMMENDATIONS  FOR  FURTHER  RESEARCH 

1.  Other  types  of  predictors,  such  as  actual  Baccalaureate 
QPR,  grades  in  mathematics  courses  or  other  courses  related 

to  Operations  Research/Systems  Analysis,  might  be  of  interest. 

2.  Check  the  predictive  validity  of  the  predictor  system 
developed  in  this  study  with  a  subsequent  sample  group,  as 
was  suggested  by  Sofge  [8], 
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APPENDIX  A 
BIOGRAPHICAL  QUESTIONNAIRE 


Biographical  Question 

1.  Did  you  receive  your  commission 
from  the  USNA? 

2.  Did  you  receive  your  commission 
through  an  ROTC  program? 

3.  Have  you  ever  been  an  enlisted 
man  in  any  service? 

4.  Is  your  rank  Navy  Lieutenant 
or  below? 

5.  Are  you  a  pilot  or  other  flight 
officer? 

6.  Are  you  a  submarine  officer? 

7.  Are  you  an  unrestricted  line 
officer? 

8.  Are  you  a  staff  officer? 

9.  Do  you  have  a  B.S.  (not  a 
B.A.)  degree? 

10.  Have  you  had  at  least  one  year 
of  college  calculus  at  an 
institution  other  than  the 
Naval  Postgraduate  School? 

11.  Do  you  speak  at  least  one 
language  other  than  English? 

12.  Do  you  have  a  master's  degree 
from  a  school  other  than  the 
Naval  Postgraduate  School? 

13.  Have  you  taken  any  graduate 
courses  other  than  at  the 
Naval  Postgraduate  School? 

14.  Have  you  ever  completed  any 
courses  at  night  school  or 
through  correspondence? 


Corr. 

Yes/No 

with  ZQPR 

29/45 

0.06 

15/59 

0.12 

26/48 

-0.19 

55/19 

0.21 

27/47 

0.04 

3/71 

-0.04 

61/13 

0.07 

11/63 

-0.10 

59/15 


68/6 
25/49 

1/73 

15/59 

32/42 


-0.03 


0.04 
0.26 

0.21 

0.22 

-0.26 
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Biographical  Questions 

15.  As  an  undergraduate  in  college, 
did  you  have  an  A  or  A- 
average? 

16.  Was  your  undergraduate  average 
in  college  below  B? 

17.  Do  you  need  to  wear  glasses 
for  reading? 

18.  Are  you  five  feet  nine  inches 
or  shorter? 

19.  Are  you  172  pounds  or  heavier? 

20.  Are  you  white  (Caucasian)? 

21.  Are  you  black  (Negro)? 

22 .  Was  a  branch  of  engineering  your 
undergraduate  major  in  college? 

23.  Are  you  Roman  Catholic? 

24.  Are  you  Protestant? 

25.  Have  you  ever  been  divorced? 

26.  Are  you  married  now? 

27.  Do  you  have  any  sons? 

28.  Do  you  have  any  daughters? 

29.  Do  you  have  any  older  brothers 
or  sisters? 

30.  Do  you  have  any  younger  brothers 
or  sisters? 

31.  Is  your  father  a  college  graduate? 

32.  Has  your  mother  ever  attended 
college? 

33.  Do  you  have  a  wife  who  is  a 
college  graduate? 

34.  Is  or  was  your  father  a  career 
military  officer? 


Corr. 
Yes/No    with  ZQPR 


7/67 


25/49 


19/55 


34/40 


29/45 


30/44 


9/65 


0.34 


-0.17 


-0.16 


17/57 

-0.17 

48/26 

0.21 

74/  0 

— 

0/74 

— 

29/45 

-0.07 

18/57 

-0.32 

47/27 

0.20 

5/69 

-0.08 

64/10 

-0.11 

40/34 

0.10 

38/36 

-0.26 

-0.00 


54/20      -0.09 
29/45     -0.02 


-0.003 


0.13 


0.11 
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Corr . 

Yes /No 

with  ZQPR 

9/69 

-0.15 

16/58 

-0.02 

64/10 

-0.19 

52/22 

0.05 

33/41 

0.14 

Biographical  Questions 

35.  Is  or  was  your  father  a  career 
enlisted  man? 

36.  Did  you  spend  more  than  one  year 
of  your  childhood  on  a  farm? 

37.  Did  you  take  a  college  prepara- 
tory program  in  high  school? 

38.  Were  you  in  the  upper  one-quarter 
of  the  college  preparatory  program 
in  high  school? 

39.  Do  you  smoke  cigarettes,  cigars 
or  pipes? 

40.  Are  you  currently  a  student  at  any 
graduate  school  other  than  the 

Naval  Postgraduate  School?  0/74 

41.  Are  you  a  student  at  the  Naval 

Postgraduate  School?  73/1       -0.02 

42.  WonlH  von  <za\r   +-Vi^+-  vou  fevDieallv 
drink  an  alcoholic  beverage  daily 
other  than  at  mealtime? 

43.  Do  you  typically  drink  more  than 
five  cups  of  coffee  a  day? 

44.  Are  you  younger  than  30  years 
of  age? 

45.  Would  you  expect  to  use  any  skills 
learned  in  graduate  school  in  sub- 
sequent assignments  in  the  Navy?       72/  2       0.07 

46.  Do  you  expect  to  use  any  graduate 
education  obtained  while  on  active 
duty  in  work  after  you  retire 

from  the  Navy?  70/  4       0.03 

47.  Do  you  wish  to  serve  in  a  billet 
requiring  the  education  that  you 
would  receive  at  a  graduate 

school?  65/  9        0.26 

48.  Would  you  prefer  to  do  your 
graduate  work  at  a  school  other 

than  the  Naval  Postgraduate  School?     29/4  5        0.26 
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20/54 

0.09 

20/54 

-0.21 

37/37 

0.27 

Biographical  Questions 

49.  Do  you  believe  that  postgraduate 
education  will  increase  your 
chances  for  promotion? 

50.  Were  you  last  designated  a  prin- 
ciple or  an  alternate  (as  opposed 
to  neither)  by  the  Postgraduate 
Selection  Board? 

51.  Have  you  ever  been  a  patrol  leader 
or  a  senior  patrol  leader  in  the 
Boy  Scouts? 

52.  Have  you  ever  been  a  Star  Scout 
or  above  in  the  Boy  Scouts? 

53.  Have  you  ever  taken  lessons  for  a 
musical  instrument  for  longer 
than  two  consecutive  years? 

54.  Do  you  now  play  a  musical  instrument? 

55.  Are  you  satisfied  with  your 
education  at  the  Naval  Post- 
graduate School? 

56 .  Are  you  or  were  you  in  the 
curriculum  of  your  first 
or  second  choice? 

57.  Were  you  ever  in  the  baccalaureate 
program? 

58.  Have  you  ever  spent  time  in  the 
engineering  science  curriculum? 

59 .  Do  you  now  like  your  degree 
curriculum? 

60.  Would  you  choose  a  different 
degree  curriculum  if  you  would 
start  over  again? 

61.  Was  at  least  part  of  your  motiva- 
tion to  remain  in  the  Navy  the 
opportunity  to  receive  postgraduate 
education? 


Corr . 
Yes/No     with  ZQPR 


57/17 

59/15 

23/51 
23/51 

34/40 
11/63 

64/10 


14/60 


40/34 


-0.19 


0.17 

0.08 
0.15 

0.02 
0.08 


64/10 

0.11 

3/71 

0.06 

35/39 

-0.02 

63/11 

-0.13 
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APPENDIX  B 
STRONG  VOCATIONAL  INTEREST  BLANK 


Mean 

Corr . 

Seal 

e 

Std. Score 

S.D. 

with  ZQPR 

1. 

Naval  Officer 

50.18 

10.52 

0.03 

2. 

Physical  Therapist 

35.58 

12.18 

-0.27 

3. 

Dentist 

21.80 

10.87 

-0.09 

4. 

Osteopath 

26.03 

11.62 

-0.21 

5. 

Veterinarian 

23.78 

9.79 

-0.37 

6. 

Physician 

25.47 

11.73 

-0.08 

7. 

Psychiatrist 

23.92 

11.04 

0.12 

8. 

Psychologist 

28.01 

9.59 

0.36 

9. 

Biologist 

26.24 

12.70 

0.14 

10. 

Architect 

24.95 

10.84 

0.11 

11. 

Mathematician 

20.50 

11.75 

0.39 

12. 

Physicist 

21.68 

13.06 

0.23 

13. 

Chemist 

30.65 

14.32 

0.26 

14. 

Engineer 

28.82 

11.40 

0.16 

15. 

Production  Manager 

39.82 

8.64 

0.00 

16. 

Army  Officer 

42.38 

10.79 

-0.06 

17. 

Air  Force  Officer 

41.78 

9.79 

-0.02 

18. 

Carpenter 

22.80 

13.08 

-0.07 

19. 

Forest  Service 

23.61 

13.59 

-0.33 

20. 

Farmer 

30.28 

11.06 

-0.20 

21. 

Math-Science  Teacher 

31.27 

9.83 

0.03 
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Mean 

Corr . 

Seal 

e 

Std. Score 

S.D. 

with  ZQPR 

22. 

Printer 

25.64 

8.88 

-0.01 

23. 

Policeman 

21.88 

7.92 

-0.31 

24. 

Personnel  Director 

30.77 

13.80 

0.07 

25. 

Public  Administrator 

41.20 

11.89 

0.06 

26. 

Rehabilitation  Coun- 

selor 

31.51 

11.52 

0.01 

27. 

YMCA  Secretary 

30.64 

13.29 

-0.36 

28. 

Recreation  Admin- 

istrator 

33.31 

12.91 

-0.25 

29. 

Social  Worker 

28.88 

12.82 

-0.04 

30.   Social  Science 
Teacher 


26.14 


10.90 


-0.10 


31.   School  Superin- 


tendent 

20.51 

10.77 

0.10 

32. 

Minister 

11.28 

13.13 

0.01 

33. 

Librarian 

25.09 

9.93 

0.24 

34. 

Artist 

22.92 

8.63 

0.07 

35. 

Music  Performer 

27.91 

8.54 

-0.07 

36. 

Music  Teacher 

23.11 

9.98 

' -0.08 

37. 

CPA  Owner 

27.65 

11.20 

0.31 

38. 

Senior  CPA 

36.35 

10.55 

0.33 

39. 

Accountant 

31.04 

10.80 

0.02 

40. 

Office  Worker 

29.62 

11.09 

-0.02 

41. 

Credit  Manager 

34.47 

13.16 

-0.06 

42. 

Chamber  of  Commerce 

36.59 

11.68 

-0.12 

43.   Business  Education 
Teacher 


32.36 


11.08 


-0.05 
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Mean 

Corr . 

Sea 

le 

Std. Score 

S.D. 

with  ZQPR 

44. 

Purchasing 

Agent 

33.22 

8.61 

0.01 

45. 

Banker 

23.77 

8.56 

0.11 

46. 

Pharmacist 

26.43 

7.98 

-0.20 

47. 

Mortician 

23.88 

7.62 

-0.31 

48. 

Sales  Manager 

26.61 

10.74 

-0.10 

49.   Real  Estate 
Salesman 


32.38 


8.14 


-0.10 


50.   Life  Insurance 


Salesman 

23.97 

9.90 

-0.20 

51. 

Advertising  Man 

25.53 

10.17 

-0.00 

52. 

Attorney 

28.01 

8.56 

0.13 

53. 

Author- Journalist 

27.58 

7.70 

0.17 

54. 

Prgci  r^onf  Manu- 

facturing Concern 

22.89 

8.83 

0.06 

55. 

Computer  Programmer 

43.19 

10.81 

0.37 

56. 

Interpreter 

28.11 

10.45 

0.21 

57. 

A-B  Doctor 

43.96 

12.98 

0.03 

58. 

Academic  Achievement 

48.54 

9.66 

0.32 

59. 

Liberal  Conservative 

43.69 

8.29 

0.16 

60. 

Masculinity- 

Femininity 

51.74 

8.14 

-0.01 

61. 

Occupational  Level 

60.50 

7.21 

0.15 

62. 

Extroversion- 

Introversion 

45.31 

19.29 

0.18 

63. 

Specialization 

Level 

42.92 

9.14 

0.32 

64. 

NROTC  Retention 

51.76 

11.26 

-0.15 

65.   Managerial 

Effectiveness 


52.01 


9.29 


0.03 
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APPENDIX  C 
THE  EFFECT  OF  SELECTION  ON  VALIDITY 

This  appendix  contains  a  statistical  explanation  of  the 
effect  of  selection  of  individuals  on  a  validity  coefficient, 
For  simplicity/  only  the  bivariate  case  will  be  considered 
here  (see  Ref .  17) . 

Suppose  that  a  regulation  had  been  in  effect  that  no  one 
with  a  baccalaureate  QPR  (BQPR)  below  2.50  would  be  admitted 
to  the  graduate  program.   Lets  denote  the  BQPR  as  variable  x 
Denote  the  performance  criterion  score  as  variable  y.   If 
selection  is  on  the  basis  of  variable  x,  the  regression  line 
of  y  on  x  will  not  be  affected ,  and  can  be  assumed  to  be  the 
same  for  the  curtailed  and  extended  distribution.   If  the 
curtailed  group  is  designated  by  x  and  y,  and  the  extended 
group  by  X  and  Y,  then  the  assumption  can  be  written  as: 


s 

r   -£  x  (1) 

xy  sx 


and 


SY 

y  -  rxy  §7  x  •  (2) 


Since  it  is  assumed  that  the  predicted  y(Y)  for  a  given 
x(X)  is  the  same  for  both  cases,  the  slopes  of  the  two 
regression  lines  are  equal,  and  can  be  written  as 
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i*  -  *XY    =C  •  (3) 


Y 

xy  sx    ... 


Moreover,  not  only  is  the  mean  of  y  for  given  x  the  same 
for  both  groups,  but  also  the  dispersion  of  y's  for  given 
x,  hence 


V l  -   rxy  "  ^Ifi-1 rxy  •         <4> 


sy 


Consider  the  case  where  one  knows  the  variance  of  x  for 
the  curtailed  group  and  the  extended  group.   In  order  to 
obtain  the  equation  for  RyY>  one  maY  solve  equation  3  for 
SY,  obtaining 

r   s  Sv 

Y     RXYSX 


and  then  substitute  this  value  for  S   in  equation  4,  obtaining 

SyV>  "  'xy   "  ^V^Y   •     <6' 


Solving  explicitly  for  R^„  gives 


Rxy      -2  .  (7) 

i  +  !|<4-  -  i) 

Sv  r 
X  xy 
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APPENDIX  D 

GRADUATE  EDUCATION  POTENTIAL  CATEGORIES 
FOR  CLASSIFICATION 


1.  Capable  of  direct  entry  into  a  technical  curriculum 

2.  Capable  of  direct  entry  into  a  non-technical  graduate 
program  not  requiring  mathematical  aptitutde. 

3.  Potentially  capable  of  entry  into  a  technical  curriculum 
after  a  refresher  course  of  3-6  months  duration. 

4.  Capable  of  direct  entry  into  a  non-technical  graduate 
program  requiring  some  mathematical  aptitude  (would 
also  meet  categories  5)  . 

5.  Capable  of  entry  into  an  updating  program  which  may  lead 
to  qualification  for  a  technical  curriculum  after  6-12 
months  study. 

6.  Capable  of  qualifying  for  category  5  by  taking  off-duty 
courses , 

7.  No  apparent  potential  for  graduate  education. 

8.  No  accredited  baccalaureate  degree.   Needs  undergraduate 
program. 


GRADUATE  EDUCATION  POTENTIAL  CLASSIFICATION  CRITERIA 
1 .   Capable  of  direct  entry  into  a  technical  curriculum 

1)  Possess  an  accredited  baccalaureat  degree  with  a 
minimum  preparation  of  mathematics  through  the 
differential  and  integral  calculus  of  several  varia- 
bles and  a  one  year  course  in  general  physics  using 
calculus  as  a  tool.   Marks  achieved  in  all  mathematics 
and  physics  courses  be  C  or  better  and  the  overall 
average  of  these  grades  at  least  2.50  on  a  scale 
having  2.00  as  C. 

2)  When  academic  credits  include  college  chemistry  or 
engineering  credits  taken  in  the  junior  or  senior 
year  an  overall  average  of  2.50  or  better  in  all 
math,  physics,  and  upper  division  enginerring  may 
be  substituted  for  the  required  overall  average  in 
math  and  physics. 
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2 .  Capable  of  direct  entry  into  a  non-technical  graduate 
program  not  requiring  mathematical  aptitude 

1)  Possess  an  accredited  baccalaureate  degree  with  an 
overall  average  of  at  least  2.75  on  a  scale  having 
2.00  as  C. 

2)  Have  an  academic  major  in  a  non-technical  subject  with 
an  average  of  at  least  3.00  in  that  subject.   A 
general  liberal  arts  degree  with  a  3.00  average  may 

be  used  as  a  substitute  if  no  major  was  pursued. 

3.  Potentially  capable  of  entry  into  a  technical  curriculum 
after  a  refresher  course  of  3-6  months  duration 

1)  Possess  an  accredited  baccalaureate  degree. 

2)  Have  passed  mathematics  courses  through  the  differ- 
ential and  integral  calculus  of  several  variables  and 
a  one  year  course  in  general  physics  using  calculus 
as  a  tool.   Have  at  least  a  2.00  average  in  all 
mathematics  and  physics  courses. 

3)  When  courses  of  2)  have  been  taken,  a  GRE  quantitative 
aptitude  score  of  550  or  higher  may  be  substituted 
for  the  2.00  average. 

4 .  Capable  of  direct  entry  into  a  non-technical  graduate 
program  requiring  some  mathematical  aptitude 

1)  Possess  an  accredited  baccalaureate  degree  with  an 
overall  average  of  at  least  2.50  on  a  scale  having 
2.00  as  a  C  average. 

2)  Have  completed  successfully  (C  grades  at  least)  a 
minimum  of  two  college  courses  in  mathematics  at 
the  level  of  college  algebra  or  higher  and  have  a 
Graduate  Record  Examination  (GRE)  Quantitative 
Aptitude  score  of  50  0  or  higher. 

3)  A  GRE  Quantitative  Aptitude  score  of  550  or  higher 
may  be  used  in  lieu  of  criteria  2) . 

5.  Entry  into  an  updating  program  which  may  lead  to  Qualifica- 
tion for  a  technical  curriculum  after  6-12  months  of  study 

1)  Possess  an  accredited  baccalaureate  degree. 

2)  Have  completed  successfully  (at  least  a  C  grade)  at 
least  one  college  mathematics  course  in  algebra, 
trigonometry,  or  math  analysis. 
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3)   When  no  college  mathematics  has  been  taken,  a 
baccalaureate  degree  with  an  overall  average  of 
2.75,  where  2.0  0  is  a  C  average,  or  a  GRE  Quanti- 
tative Aptitude  score  of  550  may  be  substituted. 

6.  Could  qualify  for  category  5  by  taking  off-duty  courses 

1)  Possess  an  accredited  baccalaureate  degree. 

2)  No  evidence  of  mathematical  inadequacy  in  form  of 
low  marks  in  courses  attempted. 

7 .  No  apparent  potential  for  graduate  education 

1)  Possess  an  accredited  baccalaureate  degree. 

2)  Not  qualified  in  categories  1-5. 

3)  Evidence  of  mathematical  inadequacy  by  low  marks 
in  courses  attempted. 

8 .  No  accredited  baccalaureate  degree 
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